Papers with optimizing large language models
NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in reinforcement learning, such as DeepSeek R1-Zero, highlight the effectiveness of incentive training, but these methods rely on external verifiers, which limits their applicability to domains like mathematics and coding, where such verifier is readily available. |
| Approach: | They propose a general reinforcement learning framework that requires only standard supervised fine-tuning data with no need for an external verifier. |
| Outcome: | The proposed framework outperforms the model of the same size distilled from large reasoning models such as DeepSeek R1 671B by 7.7%. |
MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to optimize large language models for long-context inference are inefficient and consume memory. |
| Approach: | They propose a mixed-precision quantization method via mixture of experts that inputs tokens into router chunk by chunk to reduce inference overhead. |
| Outcome: | The proposed method outperforms state-of-the-art KV cache quantization methods on multiple benchmark datasets. |
CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for proximal policy optimization discard valuable gradient signals from low-probability tokens due to the clipping mechanism. |
| Approach: | They propose an algorithm that reintroduces gradients from clipped tokens in native PPO in a gentle and bounded manner. |
| Outcome: | The proposed algorithm outperforms strong baselines on reasoning benchmarks on different model scales. |
Orchestrating Tokens and Sequences: Dynamic Hybrid Policy Optimization for RLVR (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing RLVR algorithms focus on different granularities and have complementary strengths and limitations. |
| Approach: | They propose a framework for reinforcement learning with verifiable rewards that bridges RLVR and GSPO . group-level importance ratios are used to update a policy, which preserves fine-grained credit assignment . |
| Outcome: | The proposed framework outperforms existing methods on seven reasoning benchmarks. |